Data Pipelines

Back to Home

01. Welcome
02. What is a Data Pipeline?
03. Data Validation
04. DAGs and Data Pipelines
05. Bikeshare DAG
06. Introduction to Apache Airflow
07. Demo 1: Airflow DAGs
08. Workspace Instructions
09. Exercise 1: Airflow DAGs
10. Solution 1: Airflow DAGs
11. How Airflow Works
12. Airflow Runtime Architecture
13. Building a Data Pipeline
14. Demo 2: Run the Schedules
15. Exercise 2: Run the Schedules
16. Solution 2: Run the Schedules
17. Operators and Tasks
18. Demo 3: Task Dependencies
19. Exercise 3: Task Dependencies
20. Solution: Task Dependencies
21. Airflow Hooks
22. Demo 4: Connections and Hooks
23. Exercise 4: Connections and Hooks
24. Solution 4: Connections and Hooks
25. Demo 5: Context and Templating
26. Exercise 5: Context and Templating
27. Solution 5: Context and Templating
28. Quiz: Review of Pipeline Components
29. Demo: Exercise 6: Building the S3 to Redshift DAG
30. Exercise 6: Build the S3 to Redshift DAG
31. Solution 6: Build the S3 to Redshift DAG
32. Conclusion

Back to Home

05. Bikeshare DAG

Bikeshare DAGs

Graphs

What are the two components of ALL graphs?

Cycle

Node

Edge

Direction

SOLUTION:

Node
Edge

DAGs

Which of the following are features which define a Directed Acyclic Graph?

Has Cycles

No Cycles

Nodes may have more than one edge that connects to them

Edges between nodes imply a directed relationship

SOLUTION:

No Cycles
Nodes may have more than one edge that connects to them
Edges between nodes imply a directed relationship

DAG Quiz

Which graph(s) shown above are directed acyclic graphs (DAG)?

Graph 1

Graph 2

Graph 3

SOLUTION:

Graph 2

Learn Udacity: click here to learn more :)